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Abstract.  Social  networks  are  growing  in  number  and  size,  with 
hundreds  of  millions  of  user  accounts  among  them.  One  added 
benefit  of  these  networks  is  that  they  allow  users  to  encode  more 
information  about  their  relationships  than  just  stating  who  they 
know.  In  this  work,  we  are  particularly  interested  in  trust 
relationships,  and  how  they  can  be  used  in  designing  interfaces.  In 
this  paper,  we  present  FilmTrust,  a  website  that  uses  trust  in  web- 
based  social  networks  to  create  predictive  movie  recommendations. 
Using  the  FilmTrust  system  as  a  foundation,  we  show  that  these 
recommendations  are  more  accurate  than  other  techniques  when  the 
user's  opinions  about  a  film  are  divergent  from  the  average.  We 
discuss  this  technique  both  as  an  application  of  social  network 
analysis,  as  well  as  how  it  suggests  other  analyses  that  can  be 
performed  to  help  improve  collaborative  filtering  algorithms  of  all 
types. 


1  Introduction 

Web-based  social  networks  are  growing  in  size  and  number  every  day.  A 
website  that  maintains  a  comprehensive  list  of  these  networks  shows  133 
networks  with  well  over  165,000,000  user  accounts  among  them.  Users 
spend  hours  maintaining  personal  information,  blog  entries,  and  lists  of  social 
contacts.  The  benefit  of  this  time  investment  is  vague.  While  a  small 
percentage  of  these  networks  are  dedicated  to  building  business  contacts, 
most  are  for  entertainment  purposes. 

While  entertainment  may  motivate  users  to  maintain  a  presence  in  these  web- 
based  social  networks,  there  is  great  potential  to  utilize  the  social  data  for 
enhancing  end  user  applications.  Since  the  networks  are  web-based,  the 
information  is  largely  publicly  available.  Many  of  these  networks  are 
beginning  to  output  their  members'  profiles  using  FOAF,  a  Semantic  Web 
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vocabulary  for  representing  social  networks,  means  that  the  data  is  not  only 
available  but  easily  readable  by  applications. 

One  space  that  these  social  networks  can  be  integrated  into  applications  is  in 
creating  interfaces  that  act  "intelligently"  with  respect  to  the  user's  social 
connections.  This  can  be  further  refined  by  looking  at  specific  features  of 
social  relationships.  Nearly  half  of  the  social  networks  found  in  the 
aforementioned  list  provide  some  means  for  users  to  add  information  about 
their  relationships  with  others.  This  could  include  the  type  of  relationship 
(e.g.  "friend",  "sibling",  "co-worker",  etc.),  the  strength  of  the  relationship 
(e.g.  "acquaintance",  "good  friend",  "best  friend",  etc.),  or  how  much  the 
users  trust  the  people  they  know.  Our  research  is  specifically  focused  on  this 
trust  relationship  because  it  has  many  features  that  make  it  ideal  for 
integrating  into  socially  intelligent  interfaces. 

Specifically,  we  are  will  use  social  trust  as  the  basis  for  a  recommender 
system.  For  this  technique  to  be  successful,  there  must  be  a  correlation 
between  trust  and  user  similarity.  Abdul-Rahman  and  Hailes  [1]  showed  that 
in  a  predefined  context,  such  as  movies,  users  develop  social  connections 
with  people  who  have  similar  preferences.  These  results  were  extended  in 
work  by  Ziegler  and  Lausen  [2]  that  showed  a  correlation  between  trust  and 
user  similarity  in  an  empirical  study  of  a  real  online  community. 

Furthermore,  there  is  evidence  to  support  that  users  will  prefer  systems  with 
recommendations  that  rely  on  social  networks  and  trust  relationships  over 
similarity  measures  commonly  used  for  making  recommendations.  Research 
has  shown  that  people  prefer  recommendations  from  friends  to  those  made  by 
recommender  systems  [3]  and  that  users  prefer  recommendations  from 
systems  they  trust  [4].  By  producing  recommendations  through  the  use  of 
trust  in  social  networks,  both  of  those  user  preferences  are  addressed. 
Recommendations  come  through  a  network  of  friends,  and  are  based  on  the 
explicit  trust  expressed  by  the  user. 

In  this  paper,  we  present  FilmTrust,  a  website  that  integrates  web-based 
social  networking  into  a  movie  recommender  system.  We  begin  with  a 
description  of  the  FilmTrust  website,  followed  by  an  analysis  of  its  features. 
TidalTrust,  a  trust  inference  algorithm,  is  used  as  the  basis  for  generating 
predictive  ratings  personalized  for  each  user.  The  accuracy  of  the 
recommended  ratings  is  shown  to  outperform  both  a  simple  average  rating 
and  the  ratings  produced  by  a  common  correlation-based  collaborative 
filtering  algorithm.  Theoretically  and  through  a  small  user  study,  some 


evidence  is  also  developed  that  supports  a  user  benefit  from  ordering  reviews 
based  on  the  users'  trust  preferences. 

2.  Background  and  Related  Work 

Recommender  systems  help  users  identify  items  of  interest.  These 
recommendations  are  generally  made  in  two  ways:  by  calculating  the 
similarity  between  items  and  recommending  items  related  to  those  in  which 
the  user  has  expressed  interest,  or  by  calculating  the  similarity  between  users 
in  the  system  and  recommending  items  that  are  liked  by  similar  users.  This 
latter  method  is  also  known  as  collaborative  filtering. 

Collaborative  filtering  has  been  applied  in  many  contexts,  and  FilmTrust  is 
not  the  first  to  attempt  to  make  predictive  recommendations  about  movies. 
MovieLens  [5],  Recommendz  [6],  and  Film-Conseil  [7]  are  just  a  few  of  the 
websites  that  implement  recommender  systems  in  the  context  of  films. 

Herlocker,  et  al.  [8]  present  an  excellent  overview  of  the  goals,  datasets,  and 
algorithms  of  collaborative  filtering  systems.  However,  FilmTrust  is  unlike 
the  approach  taken  in  many  collaborative  filtering  recommender  systems  in 
that  its  goal  is  not  to  present  a  list  of  good  items  to  users;  rather,  the 
recommendations  are  generated  to  suggest  how  much  a  given  user  may  be 
interested  in  an  item  that  the  user  already  found.  For  this  to  work,  there  must 
be  a  measure  of  how  closely  the  item  is  related  to  the  user's  preferences. 

Before  making  any  computations  with  trust  in  social  networks,  it  is  vitally 
important  to  know  what  trust  is.  Social  trust  depends  on  a  host  of  factors 
which  cannot  be  easily  modeled  in  a  computational  system.  Past  experience 
with  a  person  and  with  their  friends,  opinions  of  the  actions  a  person  has 
taken,  psychological  factors  impacted  by  a  lifetime  of  history  and  events 
(most  completely  unrelated  to  the  person  we  are  deciding  to  trust  or  not 
trust),  rumor,  influence  by  others'  opinions,  and  motives  to  gain  something 
extra  by  extending  trust  are  just  a  few  of  these  factors.  For  trust  to  be  used  as 
a  rating  between  people  in  social  networks,  the  definition  must  be  focused 
and  simplified.  We  adopt  this  as  the  definition  of  trust  for  our  work:  trust  in  a 
person  is  a  commitment  to  an  action  based  on  a  belief  that  the  future  actions 
of  that  person  will  lead  to  a  good  outcome.  The  action  and  commitment  does 
not  have  to  be  significant.  We  could  say  Alice  trusts  Bob  regarding  movies  if 
she  chooses  to  watch  a  film  (commits  to  an  action)  that  Bob  recommends 
(based  on  her  belief  that  Bob  will  not  waste  her  time). 


Other  work  has  touched  on  trust  in  recommender  systems,  including  [9]  and 
[10].  These  works  address  the  use  of  trust  within  systems  where  the  set  of 
commonly  rated  items  between  users  is  sparse.  That  situation  leads  to  a 
breakdown  in  correlation-based  recommender  system  algorithms,  and  their 
work  explores  how  incorporating  even  simple  binary  trust  relationships  can 
increase  the  coverage  and  thus  the  number  of  recommendations  that  can  be 
made. 

3.  Experimental  Platform:  The  FilmTrust  Website 

The  FilmTrust  system,  at  http://trust.mindswap.org/FilmTrust,  is  a  website 
that  combines  a  web-based  social  network  and  a  movie  rating  and  review 
system.  It's  membership  forms  the  basis  for  our  investigation. 


Figure  1.  A  user's  view  of  the  page  for  "A  Clockwork  Orange,"  where  the 
recommended  rating  matches  the  user's  rating,  even  though  the  average  is  quite 
different. 

3.1  Social  Networking  with  FilmTrust 

The  social  networking  component  of  the  website  allows  users  to  maintain  a 
list  of  friends  who  are  also  in  the  network.  Our  system  requires  users  to 
provide  a  trust  rating  for  each  person  they  add  as  a  friend.  When  creating  a 
trust  rating  on  the  site,  users  are  advised  to  rate  how  much  they  trust  their 


friend  about  movies.  Users  are  advised  to  consider  trust  in  this  context:  "...if 
the  person  were  to  have  rented  a  movie  to  watch,  how  likely  it  is  that  you 
would  want  to  see  that  film." 

In  the  FilmTrust  network,  relationships  can  be  one-way,  so  users  can  see  who 
they  have  listed  as  friends,  and  vice  versa  .  If  trust  ratings  are  visible  to 
everyone,  users  can  be  discouraged  from  giving  accurate  ratings  for  fear  of 
offending  or  upsetting  people  by  giving  them  low  ratings.  Because  honest 
trust  ratings  are  important  to  the  function  of  the  system,  these  values  are  kept 
private  and  shown  only  to  the  user  who  assigned  them. 

3.2  Movie  Features 

The  other  features  of  the  website  are  movie  ratings  and  reviews.  Users  can 
choose  any  film  and  rate  it  on  a  scale  of  a  half  star  to  four  stars.  They  can 
also  write  free-text  reviews  about  movies. 

Social  networks  meet  movie  information  on  the  "Ratings  and  Reviews"  page 
shown  in  Figure  1.  Users  are  shown  two  ratings  for  each  movie.  The  first  is 
the  simple  average  of  all  ratings  given  to  the  film.  The  "Recommended 
Rating"  uses  the  inferred  trust  values,  computed  with  TidalTrust  on  the  social 
network,  for  the  users  who  rated  the  film  as  weights  to  calculate  a  weighted 
average  rating.  Because  the  inferred  trust  values  reflect  how  much  the  user 
should  trust  the  opinions  of  the  person  rating  the  movie,  the  weighted  average 
of  movie  ratings  should  reflect  the  user's  opinion.  If  the  user  has  an  opinion 
that  is  different  from  the  average,  the  rating  calculated  from  trusted  friends  - 
who  should  have  similar  opinions  -  should  reflect  that  difference.  Similarly, 
if  a  movie  has  multiple  reviews,  they  are  sorted  according  to  the  inferred  trust 
rating  of  the  author.  This  presents  the  reviews  authored  by  the  most  trusted 
people  first  to  assist  the  user  in  finding  information  that  will  be  most 
relevant. 

3.3  Computing  Recommended  Movie  Ratings 

One  of  the  features  of  the  FilmTrust  site  that  uses  the  social  network  is  the 
"Recommended  Rating"  feature.  As  Figure  1  shows,  users  will  see  this  in 
addition  to  the  average  rating  given  to  a  particular  movie. 

The  "Recommended  Rating"  is  personalized  using  the  trust  values  (direct  or 
inferred)  that  the  user  has  the  people  who  have  rated  the  film  (the  raters).  If  a 
user  Alice  has  directly  assigned  a  trust  rating  to  another  user,  Bob,  then  the 
trust  value  is  known.  If  Alice  has  not  rated  Bob,  we  need  to  infer  how  much 
she  might  trust  him.  Trust  inference  systems  are  a  growing  area  of  interest.  In 


this  application,  we  utilize  TidalTrust,  a  breadth  first  search-based  algorithm 
that  outputs  an  inferred  trust  value  by  finding  paths  form  Alice  to  Bob  and 
composing  the  trust  values  found  along  those  paths.  Details  of  that  algorithm 
are  beyond  the  scope  of  this  paper,  but  can  be  found  in  [1 1]  and  [12]. 

To  compute  the  recommended  movie  rating,  the  FilmTrust  system  first 
searches  for  raters  who  the  user  knows  directly.  If  there  are  no  direct 
connections  from  the  user  to  any  raters,  the  system  moves  one  step  out  to  find 
connections  from  the  user  to  raters  of  path  length  2.  This  process  repeats  until 
a  path  is  found.  The  opinion  of  all  raters  at  that  depth  are  considered.  Then, 
using  TidalTrust,  the  trust  value  is  calculated  for  each  rater  at  the  given 
depth.  Once  every  rater  has  been  given  an  inferred  trust  value,  only  the  ones 
with  the  highest  trust  values  will  be  selected;  this  is  done  by  simply  finding 
the  maximum  trust  value  calculated  for  each  of  the  raters  at  the  selected 
depth,  and  choosing  all  of  the  raters  for  which  that  maximum  value  was 
calculated.  Finally,  once  the  raters  have  been  selected,  their  ratings  for  the 
movie  (in  number  of  stars)  are  averaged.  For  the  set  of  selected  nodes  S,  the 
recommended  rating  r  from  node  s  to  movie  m  is  the  average  of  the  movie 
ratings  from  nodes  in  S  weighted  by  the  trust  value  t  from  s  to  each  node: 


iBS 


This  average  is  rounded  to  the  nearest  half-star,  and  that  value  becomes  the 
"Recommended  Rating"  that  is  personalized  for  each  user. 

As  a  simple  example,  consider  the  following: 

•  Alice  trusts  Bob  9 

•  Alice  trusts  Chuck  3 

•  Bob  rates  the  movie  "Jaws"  with  4  stars 

•  Chuck  rates  the  movie  "Jaws"  with  2  stars 


Then  Alice's  recommended  rating  for  "Jaws"  is  calculated  as  follows: 


t  *  r  4-  /  r 

Alice -> Bob  Bob->Jaws  Atice->Chuck  Chuck- Jaws 

^ Alice->Bob  ^ Alice ->Chuck 


9*4  +  3*2  42 


9+3 


12 


=  3.5 


4  Experimental  Setup  and  Design 


We  are  interested  in  knowing  if  the  trust-based  movie  ratings  offer  a  benefit 
to  the  users,  and  if  so,  in  what  instances.  To  check  this,  we  used  the  data 
users  have  entered  into  the  FilmTrust  system. 

4.1  Experimental  Setup  and  Design 

The  FilmTrust  user  base  was  used  as  the  foundation  for  our  experiments. 
When  joining  the  network,  members  were  informed  that  their  participation 
was  part  of  a  research  project,  and  they  consented  to  allow  their  data  to  be 
used  within  experiments.  The  system  has  just  over  500  members. 

Members  were  invited  by  friends  who  were  already  members  and  also  found 
out  about  the  website  from  postings  in  movie  related  forums.  There  is  a 
strong  Semantic  Web  component  to  the  website  (social  network  and  movie 
information  is  all  published  in  RDF),  so  members  were  frequently  recruited 
from  this  circle  of  interest.  Subjects  ranged  in  age  from  14  to  79,  with  an 
average  age  of  32.  Subjects  were  29%  female  and  71%  male. 


Figure  2.  A  visualization  of  the  FilmTrust  social  network 


FilmTrust  users  have  created  approximately  11,250  ratings  and  movie 
reviews  for  1,250  different  movies.  For  each  movie,  the  average  rating  was 
computed  as  the  simple  average  of  all  the  ratings  assigned  to  the  film.  To 
ensure  that  a  common  set  of  movies  were  rated,  users  were  asked  during  the 
registration  process  to  assign  ratings  to  any  movies  they  had  seen  the  top  50 
films  AFI  Top  100  Films  list  [13]. 

Not  all  of  these  members  are  connected  into  the  social  network. 
Approximately  150  of  the  500  members  do  not  have  any  social  connections. 
Their  participation  is  limited  to  entering  data  about  movies.  Of  the  members 
who  are  participating  in  the  social  network,  most  are  connected  into  a  strong 
central  core,  with  a  scattering  of  small  groups.  A  spring-embedded 
visualization  of  the  social  network  structure  is  shown  in  Figure  2 


4.2  Experimental  Results 

To  determine  the  effectiveness  of  the  recommended  ratings,  we  compare  to 
see  how  closely  they  resemble  the  actual  ratings  a  user  has  assigned  to  a  film. 
We  use  the  absolute  difference  between  the  recommended  rating  and  actual 
rating  as  our  measure.  In  this  analysis,  we  also  compare  the  user's  rating  with 
the  average  rating  for  the  movie,  and  with  a  recommended  rating  generated 
by  an  automatic  collaborative  filtering  (ACF)  algorithm.  There  are  many 
ACF  algorithms,  and  one  that  has  been  well  tested,  and  which  is  used  here,  is 
the  classic  user-to-user  nearest  neighbor  prediction  algorithm  based  on 
Pearson  Correlation  [5],  If  the  trust-based  method  of  calculating  ratings  is 
best,  the  difference  between  the  personalized  rating  and  the  user's  actual 
rating  should  be  significantly  smaller  than  the  difference  between  the  actual 
rating  and  the  average  rating.  We  label  these  measures  as  follows: 

•dr  -  the  absolute  difference  between  the  user's  rating  and  the  trust-based 
recommended  rating 

•da  -  the  absolute  difference  between  the  user's  rating  and  the  average  rating 
•  dcf  -  the  absolute  difference  between  the  user's  rating  and  the 
recommended  rating  from  the  collaborative  filtering  algorithm 

Because  the  recommended  ratings  rely  on  using  the  trust  values  in  the  social 
network,  we  were  only  able  to  make  this  comparison  for  users  with  social 
connections,  approximately  350  of  the  500  total  users.  For  each  user,  we 
selected  each  movie  and  computed  the  d  values.  In  the  end,  we  made 
comparisons  for  a  total  of  1152  movies. 


On  first  analysis,  it  did  not  appear  that  that  the  trust-based  ratings  that 
utilized  the  social  network  were  any  more  accurate  than  average.  The 
difference  between  the  actual  rating  and  the  recommended  rating  (dr)  was  not 
statistically  different  than  the  difference  between  the  user's  actual  rating  and 
the  average  rating  (da).  The  difference  between  a  user's  actual  rating  of  a  film 
and  the  ACF  calculated  rating  (dcf)  also  was  not  better  than  da  in  the  general 
case.  A  close  look  at  the  data  suggested  why.  Most  of  the  time,  the  majority 
of  users  actual  ratings  are  close  to  the  average.  This  is  most  likely  due  to  the 
fact  that  the  users  in  the  FilmTrust  system  had  all  rated  the  AFI  Top  50 
movies,  which  received  disproportionately  high  ratings.  A  random  sampling 
of  movies  showed  that  about  50%  of  all  ratings  were  within  the  range  of  the 
mean  +/-  a  half  star  (the  smallest  possible  increment).  For  users  who  gave 
these  near-mean  rating,  a  personalized  rating  could  not  offer  much  benefit 
over  the  average. 

However,  one  of  our  initial  motivations  for  creating  the  trust-based 
recommended  ratings  was  to  help  people  who  disagree  with  the  average.  In 
those  cases,  the  personalized  rating  should  give  the  user  a  better 
recommendation,  because  we  expect  the  people  they  trust  will  have  tastes 
similar  to  their  own  [10]. 

To  see  this  effect,  da,  dcf,  and  dr  were  calculated  with  various  minimum 
thresholds  on  the  da  value;  that  is,  the  user's  rating  had  to  be  at  least  da  stars 
different  from  the  average  rating.  If  the  recommended  ratings  do  not  offer  a 
benefit  over  the  average  rating,  the  dr  values  will  increase  at  the  same  rate  the 
da  values  do.  The  experiment  was  conducted  by  limiting  da  in  increments  of 
0.5.  The  first  set  of  comparisons  was  taken  with  no  threshold,  where  the 
difference  between  da  and  dr  was  not  significant.  As  the  minimum  da  value 
was  raised  it  selected  a  smaller  group  of  user-film  pairs  where  the  users  made 
ratings  that  differed  increasingly  with  the  average.  Obviously,  we  expect  the 
average  da  value  will  increase  by  about  0.5  at  each  increment,  and  that  it  will 
be  somewhat  higher  than  the  minimum  threshold.  The  real  question  is  how 
the  dr  will  be  impacted.  If  it  increases  at  the  same  rate,  then  the 
recommended  ratings  do  not  offer  much  benefit  over  the  simple  average.  If  it 
increases  at  a  slower  rate,  that  means  that,  as  the  user  strays  from  the  average, 
the  recommended  rating  more  closely  reflects  their  opinions.  Figure  3 
illustrates  the  results  of  these  comparisons. 

Notice  that  the  da  value  increases  about  as  expected.  The  dr,  however,  is 
clearly  increasing  at  a  slower  rate  than  da.  At  each  step,  as  the  lower 
threshold  for  da  is  increased  by  0.5,  dr  increases  by  an  average  of  less  than 
0.1.  A  two-tailed  t-test  shows  that  at  each  step  where  the  minimum  da 


threshold  is  greater  than  or  equal  to  0.5,  the  recommended  rating  is 
significantly  closer  to  the  user's  actual  rating  than  the  average  rating  is,  with 
p<0.01.  For  about  25%  of  the  ratings  assigned,  da<0.5,  and  the  user's  ratings 
are  about  the  same  as  the  mean.  For  the  other  75%  of  the  ratings,  <3a>0.5,  and 
the  recommended  rating  significantly  outperforms  the  average. 

As  is  shown  in  Figure  3,  dcf  closely  follows  da.  For  da<l,  there  was  no 
significant  difference  between  the  accuracy  of  the  ACF  ratings  and  the  trust- 
based  recommended  rating.  However,  when  the  gap  between  the  actual  rating 
and  the  average  increases,  for  da>=l,  the  trust-based  recommendation 
outperforms  the  ACF  as  well  as  the  average,  with  p<0.01.  Because  the  ACF 
algorithm  is  only  capturing  overall  correlation,  it  is  tracking  the  average 
because  most  users'  ratings  are  close  to  the  average. 

Figure  1  illustrates  one  of  the  examples  where  the  recommended  value 
reflects  the  user's  tastes.  "A  Clockwork  Orange"  is  one  of  the  films  in  the 
database  that  has  a  strong  collective  of  users  who  hated  the  movie,  even 
though  the  average  rating  was  3  stars  and  many  users  gave  it  a  full  4-star 
rating.  For  the  user  shown,  da=2.5  -  a  very  high  value  -  while  the 
recommended  rating  exactly  matches  the  user's  low  rating  of  0.5  stars.  These 
are  precisely  the  type  of  cases  that  the  recommended  rating  is  designed  to 
address. 

Thus,  when  the  user's  rating  of  a  movie  is  different  than  the  average  rating,  it 
is  likely  that  the  recommended  rating  will  more  closely  reflect  the  user's 
tastes.  When  the  user  has  different  tastes  than  the  population  at  large,  the 
recommended  rating  reflects  that.  When  the  user  has  tastes  that  align  with  the 
mean,  the  recommended  rating  also  aligns  with  the  mean.  Based  on  these 
findings,  the  recommended  ratings  should  be  useful  when  people  have  never 
seen  a  movie.  Since  they  accurately  reflect  the  users'  opinions  of  movies  they 
have  already.  Because  the  rating  is  personalized,  originating  from  a  social 
network,  it  is  also  in  line  with  other  results  [3,4]  that  show  users  prefer 
recommendations  from  friends  and  trusted  systems. 


0  values  over  minimum  da 


Figure  3.  The  increase  in  9  as  the  minimum  9a  is  increased.  Notice  that  the  ACF- 
based  recommendation  (9cf)  closely  follows  the  average  (9a).  The  more  accurate 
Trust-based  recommendation  (9r)  significantly  outperforms  both  other  methods. 


One  potential  drawback  to  creating  recommendations  based  solely  on 
relationships  in  the  social  network  is  that  a  recommendation  cannot  be 
calculated  when  there  are  no  paths  from  the  user  to  any  people  who  have 
rated  a  movie.  This  case  is  rare,  though,  because  as  long  as  just  one  path  can 
be  found,  a  recommendation  can  be  made.  In  the  FilmTrust  network,  when 
the  user  has  made  at  least  one  social  connection,  a  recommendation  can  be 
made  for  95%  of  the  user-movie  pairs. 

In  addition,  the  quality  of  results  is  dependent  on  users  assigning  accurate 
trust  values  to  people  in  the  system.  If  the  trust  ratings  become  too  noisy, 
they  cease  to  be  an  effective  grounds  for  making  recommendations.  The 
FilmTrust  system  is  still  relatively  small  compared  to  other  social  networks, 
which  can  have  tens  of  thousands  up  to  millions  of  members.  It  remains  to  be 
seen  how  well  this  technique  will  work  on  larger  networks.  We  have  not  yet 
been  given  access  to  trust  values  in  some  of  the  larger  networks,  and  that 
analysis  will  be  necessary  to  verify  that  user  behavior  will  support  our 
approach. 

4.3  Presenting  Ordered  Reviews 

In  addition  to  presenting  personalized  ratings,  the  experience  of  reading 
reviews  is  also  personalized.  The  reviews  are  presented  to  the  user  in  order  of 


the  trust  value  of  the  author,  with  the  reviews  from  the  most  trustworthy 
people  appearing  at  the  top,  and  those  from  the  least  trustworthy  at  the 
bottom.  The  expectation  is  that  the  most  relevant  reviews  will  come  from 
more  trusted  users,  and  thus  they  will  be  shown  first. 
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Ratings  of  E.T.  the  Extra-Terrestrial  (1982) 


Number  of  Ratings  301 

Average  User  Rating 
Your  Recommended  Rating  |  »♦»■ 
Your  Rating  |  ♦♦ 


Reviews  of  E.T.  the  Extra-Terrestrial  (1982) 

Totally  over-rated  movie,  lots  of  hype  for  little  gain  -  sorry,  other  than  a 
couple  cute  effects  and  one  line  ("go  home")  it's  just  not  worth  it  -  of  course,  it 
was  Drew  Barrymore's  best  film :-) 

-  by  Jim  Hendlcr 
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I  loved  it  when  I  was  12.  But  it  seems  a  little  sappy  in  retrospect. 

-  by  Aaron  Marines 


A  very  fun  movie,  great  acting 

-  by  Edna  Ruckhaus 
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No  age  for  watching,  and  will  never  be  old.  It  is  not  just  a  cute  movie,  or  a 
one  who  treat  children  as  stupid-littlc-pcoplc-who-will-watch-anything,  it 
really  have  a  story  behind.  The  best  movie  of  Drew  Barrymore,  i  agree,  but 
one  of  the  best  movies  ever 

-  by  Paula  Groff 
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Figure  4.  Reviews  of  "E.T."  sorted  according  to  the  trust  value  that  the  user  has  for 
each  author.  Note  that  the  ratings  of  the  ordering  also  corresponds  to  how  closely  the 
reviewers'  ratings  of  the  film  correspond  with  the  user's  rating,  even  though  that  was 
not  considered  in  choosing  the  ordering. 


For  example,  Figure  4  shows  the  reviews  of  "E.T."  ordered  for  a  user.  The 
reviews  from  more  trusted  people  appear  at  the  top  of  the  list,  and  less  trust 
people  are  further  down.  Notice  that  the  user's  rating  is  2  stars.  Even  though 
the  reviewers'  rating  were  not  considered  in  the  ordering,  they  are  ordered  as 
well;  the  reviewers  with  ratings  that  most  closely  match  the  user's  rating  are 
shown  first,  and  the  reviews  further  down  in  the  list  are  different  from  the 


user.  This  supports  the  premise  that  ordering  reviews  by  trust  rating  will 
show  users  the  opinions  more  relevant  to  their  own  perspective  first. 

Unlike  the  personalized  ratings,  measuring  the  accuracy  of  the  review  sort  is 
not  possible  without  requiring  users  to  list  the  order  in  which  they  suggest  the 
reviews  appear.  Without  performing  that  sort  of  analysis,  much  of  the 
evidence  presented  so  far  supports  this  ordering.  That  definition  also  supports 
the  ordering  of  reviews.  Trust  with  respect  to  movies  means  that  the  user 
believes  that  the  trusted  person  will  give  good  and  useful  information  about 
the  movies.  The  analysis  also  suggests  that  more  trusted  individuals  will  give 
more  accurate  information.  It  was  shown  there  that  trust  correlates  with  the 
accuracy  of  ratings.  Reviews  will  be  written  in  line  with  ratings  (i.e.  a  user 
will  not  give  a  high  rating  to  a  movie  and  then  write  a  poor  review  of  it),  and 
since  ratings  from  highly  trusted  users  are  more  accurate,  it  follows  that 
reviews  should  also  be  more  accurate. 

A  small  pilot  study  with  9  subjects  was  run  on  the  FilmTrust  network. 
Subjects  were  shown  the  reviews  for  a  movie  and  asked  to  order  them 
according  to  how  closely  they  matched  the  subject's  opinion.  This  was 
frequently  identical  to  the  ordering  based  on  trust  value,  and  the  variations 
the  did  occur  were  typically  small.  When  shown  the  trust-based  ordering,  our 
small  sample  of  users  had  a  universally  strong  positive  reaction.  While  these 
preliminary  results  show  a  strong  user  preference  for  reviews  ordered  by  the 
trustworthiness  of  the  rater,  this  study  must  be  extended  and  refined  in  the 
future  to  validate  these  results. 

5.  Conclusions  and  Discussion 

Within  the  FilmTrust  website,  trust  in  social  networks  has  been  used  as  the 
foundation  for  generating  predictive  movie  recommendations,  The  accuracy 
of  the  trust-based  predictive  ratings  in  this  system  is  significantly  better  than 
the  accuracy  of  a  simple  average  of  the  ratings  assigned  to  a  movie.  The  trust 
system  also  outperforms  the  recommended  ratings  from  a  Person-correlation 
based  recommender  system. 

Overall,  we  believe  that  FilmTrust  is  an  example  of  how  trust  and  social 
networks  can  be  exploited  to  refine  the  user  experience.  By  using  the  social 
network  data  in  computations,  the  efforts  users  are  already  putting  to  web- 
based  socializing  can  be  harnessed  to  enhance  existing  tools.  The  purpose  of 
this  work  is  not  necessarily  to  replace  more  traditional  methods  of 
collaborative  filtering.  It  is  very  possible  that  a  combined  approach  of  trust 
with  correlation  weighting  or  another  form  of  collaborative  filtering  may 


offer  equal  or  better  accuracy,  and  it  will  certainly  allow  for  higher  coverage. 
However,  these  results  clearly  show  that,  in  the  FilmTrust  network,  basing 
recommendations  on  the  expressed  trust  for  other  people  in  the  network 
offers  significant  benefits  for  accuracy. 

There  are  many  future  steps  for  both  refining  this  work  and  taking  it  in  future 
directions.  One  step  is  to  do  a  deeper  comparison  with  the  most  advanced 
collaborative  filtering  algorithms.  We  have  chosen  a  common,  basic 
algorithm  for  comparison  in  this  study.  Since  our  goal  was  not  to  out  perform 
collaborative  filtering  techniques,  but  rather  to  show  that  the  trust-based 
recommendations  were  useful. 

One  current  project  we  have  underway  is  investigating  how  users  assign  trust 
in  social  networks.  The  results  presented  here  show  that  it  is  not  merely 
correlation  of  opinions;  if  that  were  the  case,  we  would  have  seen  equivalent 
performance  between  the  trust-based  recommendations  and  the  collaborative 
filtering  recommendations.  We  believe  that  users  assign  trust  based  more  on 
agreement  on  outliers,  rather  than  on  overall  agreement.  For  example,  say 
Bob  and  Alice  both  hated  the  "Lord  of  the  Rings"  movies,  loved  "From  Justin 
to  Kelly"  ,  but  otherwise  had  a  large  variation  in  movies  about  which  they  are 
less  enthusiastic.  We  believe  that  they  may  trust  each  other  more  than  they 
would  trust  someone  with  a  higher  overall  correlation  but  who  disagreed 
about  "Lord  of  the  Rings"  and  "From  Justin  to  Kelly".  Understanding  which 
features  of  user  profiles  correlate  to  higher  trust  values  will  give  social 
insight,  but  it  also  suggests  how  different  features  of  profile  similarity  can  be 
incorporated  into  collaborative  filtering  algorithms  to  improve  their  accuracy 
even  when  social  networks  are  unavailable. 
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